feature: add export command that exports mails to markdown#1
Merged
Conversation
- Add convertPlainText(): appends trailing \ between consecutive prose lines to preserve hard line breaks in CommonMark - Normalise ASCII divider lines (---, ===) to thematic breaks; inserts blank line before divider so preceding text is not parsed as a Setext heading - Escape leading # on ATX heading lines (\# text) so plain text content is not promoted to a heading - HTML-escape < and > throughout - Detect manually typed list lines (- / * / N.) as block elements that never receive trailing \ Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Unwrap layout tables (no <th> / role="presentation") to prose; keep data tables as GFM pipe tables - Strip invisible email spacer characters (soft hyphen, ZWS etc.) in preprocessHtml - Post-process htmlToMarkdown output: replace Unicode non-breaking spaces, trim trailing whitespace per line, collapse 3+ newlines to 2, convert Turndown-escaped visual bullets (\* / \-) to real list items - Plain text: keep > unchanged so quoted lines become Markdown blockquotes; classify blockquote lines as isBlock so they neither receive nor trigger hard-break markers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove any <img> whose src is not a cid: URL — eliminates logos, layout images, and tracking pixels. Inline attachments (cid:) are preserved. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add U+034F (combining grapheme joiner) to invisible-char strip in preprocessHtml — used by newsletters to pad email preview text - Remove Markdown links with empty text after conversion, which appear when stripped images were wrapped in <a> tags (tracking/logo anchors) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- from/to use "Name <email>" strings instead of nested name/email objects - attachments use "filename (N bytes)" strings instead of nested objects Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Newsletters often use <th> for multi-column image/text grids. A <th> containing block-level elements (img, div, p, table, headings, etc.) is a layout cell, not a data header — unwrap to prose instead of forcing a broken GFM pipe table. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Tab-only lines used as separators were not classified as empty, causing the preceding content line to receive a spurious hard-break marker \. Also trims trailing whitespace from all non-empty plain text lines. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add formats list to frontmatter (plaintext, html, or both) - Swap body order: plain text first (canonical), HTML conversion second - Replace triple-divider with labelled separator block so AI agents and humans can identify the boundary and understand the HTML section is automatically converted - Rename receivedDate frontmatter field to received Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…order Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
backmail export— HTML/plain-text archive export to MarkdownAdds a new
backmail exportcommand that converts the entire.emlarchive into human- and AI-readable Markdown files, one per message, mirroring the folder structure underexport/.What it does
export/<folder>/<filename>.mdwith YAML frontmatter + subject heading + bodyOutput format
Each file looks like:
Conversion quality improvements
A significant portion of this PR is hardening the conversion pipeline against the messy reality of HTML email:
<th>with inline-only content<th>cells used for two-column newsletter gridscellHasBlockContentcheck — if a<th>contains<div>,<img>,<p>etc. it's layout, not a headeraddMarkdownHardBreakspass adds\between consecutive prose lines<img>exceptcid:inline attachments ,͏,­) surviving as invisible junkpreprocessHtmland as a post-Turndown pass (entities are decoded inside Turndown's DOM parser)[](url)links from stripped image anchors\markers on quoted reply lines (>)>lines as block elements — no hard-break markers\classifyLine#, normalise---/===lines to thematic breaksTests
tests/integration/export.test.ts— full export of a temp archive across two folders, idempotency,--only-folder,--exclude-folder